Transmission apparatus, transmission method, reception apparatus, and reception method

ABSTRACT

There is provided a transmission apparatus including: a processing unit that performs processing of mixing, at a mixing rate independent for each frame, image data in peripheral frames with image data in each frame of first moving image data at a first frame rate and obtains second moving image data at the first frame rate. At least image data in a frame corresponding to a second frame rate that is lower than the first frame rate in the image data in each frame that forms the second moving image data is brought into a state in which the image data is mixed with the image data in the peripheral frames.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2017-085730 filed Apr. 24, 2017, the entire contents ofwhich are incorporated herein by reference.

TECHNICAL FIELD

The present technology relates to a transmission apparatus, atransmission method, a reception apparatus, and a reception method, andmore particularly to a transmission apparatus that transits moving imagedata at a high frame rate and the like. Background Art

In recent years, a camera that performs high frame rate imaging with ahigh-speed frame shutter is known. For example, a normal frame rate is60 Hz, 50 Hz, or the like while the high frame rate is a frame rate thatis several times, several tens of times, or further several hundreds oftimes as high as the normal frame rate.

In a case in which a high frame rate service is performed, it isconsidered that moving image data captured by a camera with a high-speedframe shutter is converted into and transmitted as a moving imagesequence at a lower frequency than that of the moving image data.However, the image captured with the high-speed frame shutter has aneffect that moving blur is improved and image quality with highsharpness is realized while the image has an element that leads to aproblem in a traditional frame inter-polation technology in terms of theimage quality on a reception and reproduction side that displays themoving image sequence at a lower frame rate than the distributed highframe rate.

In the frame interpolation using an image with high sharpness capturedwith the high-speed frame shutter, there is a large difference between acase in which motion vector search is adapted and in a case in which themotion vector search is not adapted. Therefore, the difference betweenboth the cases is displayed as significant degradation in the imagequality. High load computation is necessary to improve accuracy of themotion vector search at the time of the frame interpolation, whichaffects receiver cost.

The applicant previously proposed a technology of converting an imagematerial captured with a high-speed frame shutter and causing atraditional receiver that performs decoding at a normal frame rate todisplay the image with predetermined or higher image quality (see PTL1).

CITATION LIST Patent Literature

PTL 1: International Publication No. 2015/076277

SUMMARY OF INVENTION Technical Problem

It is desirable to satisfactorily transfer moving image data at a normalframe rate and a high frame rate.

Solution to Problem

According to an embodiment of the present technology, there is provideda transmission apparatus that performs processing of mixing, at a mixingrate for each frame, a frame of first video data with one or moreperipheral frames of the first video data and obtains second video dataat a first frame rate. The second video data includes framescorresponding to a second frame rate that is lower than the first framerate, where the frames corresponding to the second frame rate beingmixed with the peripheral frames. The transmission apparatus encodes theframes corresponding to the second frame rate to obtain a basic streamand encodes remaining frames of the second video data to obtain anextended stream. The transmission apparatus then inserts informationabout the mixing rate of corresponding frames into the basic stream andthe extended stream in association with the respective frames, andtransmits the basic stream and the extended stream into which theinformation about the mixing rate has been inserted.

According to an embodiment of the present technology, the basic streamand the extended stream have a Network Abstraction Layer (NAL) unitstructure, and the transmission apparatus inserts a SupplementalEnhancement Information (SEI) NAL unit with the information about themixing rate into the basic stream and the extended stream. In anembodiment, the first frame rate is 120 Hz or 240 Hz, and the secondframe rate is 60 Hz.

Information about the mixing rate, which is inserted into the basicstream and the extended stream, may include configuration information ofa filter used to perform the mixing processing.

The information about the mixing rate may also include mixing refreshinformation indicating a number of frames until mixing refresh that doesnot use a temporally previous frame is performed.

The information about the mixing rate may also include refresh flaginformation indicating whether or not a respective frame is a target ofthe mixing refresh. The information about the mixing rate may alsoinclude head flag information indicating whether or not a respectiveframe corresponds to the second frame rate.

According to an embodiment of the present technology, at least the framecorresponding to the second frame rate (normal frame rate) in each framethat forms the second video data at the first frame rata (high framerate) is mixed with the peripheral frames and is in a state in which anaperture ratio is raised, and the basic stream obtained by encoding theframe corresponding to the second frame rate (normal frame rate) istransmitted as described above.

Therefore, in a case of a receiver that has decoding ability with whichthe video data at the second frame rate (normal frame rate) can beprocessed, it is possible to display a smooth image as a moving image byprocessing the basic stream and obtaining the second frame rate and toavoid occurrence of a problem in image quality in a frame interpolationprocessing based on low load computation in display processing.

In addition, according to an embodiment of the present technology, theextended stream obtained by encoding the remaining frames is obtainedalong with the basic stream, and the information about the mixing rateof the corresponding frames is inserted into the basic stream and theextended stream in association with the respective frames, and theextended stream is then transmitted. Therefore, in a case of a receiverthat has decoding ability with which the video data at the first framerate (high frame rate) can be processed, it is possible to easily obtainthe mixing-released video data at the first frame rate on the basis ofthe information about the mixing rate in each frame and tosatisfactorily display the moving image at the first frame rate.

In addition, according to another embodiment of the present technology,there is provided a reception apparatus that receives a basic stream andan extended stream, which are obtained by performing processing ofmixing, at a mixing rate for each frame, a frame of first video datawith one or more peripheral frames of the first video data. Thereception apparatus obtains second video data at a first frame rate, thesecond video data including frames corresponding to a second frame ratethat is lower than the first frame rate.

The reception apparatus mixes the frames corresponding to the secondframe rate with the peripheral frames and encodes the framescorresponding to the second frame rate to obtain the basic stream. Thereception apparatus encodes remaining frames of the second video data toobtain the extended stream. Information about the mixing rate ofcorresponding frames is included in the basic stream and the extendedstream in association with the respective frames.

The reception apparatus further decodes, based on a frame ratecapability of a display connected to the reception apparatus, either thebasic stream to obtain frames at the second frame rate or the basicstream and the extended stream to obtain the second video data, and, inthe latter case, obtains mixing-released video data at the first framerate by performing back mixing processing on the second video data on abasis of the information about the mixing rate.

According to an embodiment of the present technology, the video data atthe second frame rate (normal frame rate) is obtained by processing onlythe basic stream in a case in which there is decoding ability with whichthe video data at the second frame rate (normal frame rate) as describedabove. Since the image data in each frame that forms the video data atthe second frame rate (normal frame rate) is mixed with the peripheralframes, and a shutter aperture ratio is raised, it is possible todisplay a smooth image as a moving image and to avoid occurrence of aproblem in image quality in the frame interpolation processing based onlow load computation in the display processing.

In addition, according to an embodiment of the present technology, boththe basic stream and the extended stream are processed to obtain thevideo data at the first frame rate (high frame rate) after the mixingprocessing, and further, the back mixing processing is performed on thebasis of the information about the mixing rate in each frame to obtainthe mixing-released video data at the first frame rate (normal framerate) in a case in which there is decoding ability with which the videodata at the first frame rate (high frame rate) can be processed.Therefore, it is possible to satisfactorily display the moving image atthe first frame rate (high frame rate).

In addition, according to another embodiment of the present technology,there is provided a reception apparatus that acquires second video dataobtained by performing processing of mixing, at a mixing rate for eachframe, a frame of first video data with one or more peripheral frames ofthe first video data. The reception apparatus then transmits the secondvideo data and information about the mixing rate in each frame to anexternal device via a transfer path.

According to an embodiment of the present technology, synchronizationframe information indicating whether or not it is necessary tosynchronize with a next video frame is included in the information aboutthe mixing rate. The reception apparatus respectively inserts theinformation about the mixing rate in each frame into a blanking periodof each frame of the second video data and transmits the second videodata.

According to an embodiment of the present technology, the receptionapparatus performs back mixing processing on each frame of the secondvideo data on a basis of the information about the mixing rate to obtainthird video data. The reception apparatus then transmits the third videodata instead of the second video data when the external device does nothave a function of the back mixing processing.

According to an embodiment, the second video data has a first framerate, the second video data includes frames corresponding to a secondframe rate that is lower than the first frame rate, and the framescorresponding to the second frame rate are mixed with the peripheralframes.

In addition, in an embodiment of the present technology, for example,the reception apparatus then transmits fourth video data that includesthe frames corresponding to the second frame rate instead of the secondvideo data when a frame rate at which display is able to be performed bythe external device is the second frame rate.

In addition, according to another embodiment of the present technology,there is provided a reception apparatus that receives second video dataobtained by performing processing of mixing, at a mixing rate for eachframe, a frame of first video data with one or more peripheral frames ofthe first video data. The reception apparatus also receives informationabout a mixing rate in each frame from an external device via a transferpath. The reception apparatus then obtains mixing-released video data byperforming back mixing processing on each frame of the second video dataon a basis of the information about the mixing rate.

According to an embodiment of the present technology, the receptionapparatus also receives information about a mixing rate in each framefrom an external device via a transfer path. The reception apparatusthen obtains mixing-released video data by performing back mixingprocessing on each frame of the second video data on a basis of theinformation about the mixing rate.

According to an embodiment of the present technology, the informationabout the mixing rate in each frame is received along with the secondvideo data after the mixing processing from the external device, and themixing-released video data is obtained by performing the back mixingprocessing on each frame of the second video data on the basis of theinformation about the mixing rate. Therefore, it is possible toappropriately obtain the video data with accuracy that is similar tothat before the mixing processing and to satisfactorily display themoving image.

Advantageous Effects of Invention

According to an embodiment of the present technology, it is possible tosatisfactorily transfer moving image data at the normal frame rate andthe high frame rate. In addition, the advantages described herein arenot necessarily limited, and any of the advantages described in thepresent disclosure may be achieved.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of atransmission and reception system according to an embodiment.

FIG. 2 is a diagram illustrating an example of a basic stream at 60 Hzthat is obtained by performing mixing processing on moving image data at120 Hz, and an extended stream at +60 Hz.

FIG. 3 is a diagram illustrating an example of a basic stream at 60 Hzthat is obtained by performing mixing processing on moving image data at240 Hz, and an extended stream added thereto.

FIG. 4 is a diagram illustrating an outline of processing performed by atransmission apparatus and a television receiver.

FIG. 5 is a diagram schematically illustrating an example of mixing on atransmission side and back mixing on a reception side.

FIG. 6 is a diagram schematically illustrating another example of themixing on the transmission side and the back mixing on the receptionside.

FIG. 7 is a block diagram illustrating a configuration example of thetransmission apparatus.

FIG. 8 is a block diagram illustrating a configuration example of apre-processor that performs the mixing processing on the transmissionside.

FIG. 9 is a block diagram illustrating another configuration example ofthe preprocessor that performs the mixing processing on the transmissionside.

FIG. 10 is a block diagram illustrating a configuration example of apost-processor that performs back mixing processing on the receptionside.

FIG. 11 is a block diagram illustrating another configuration example ofthe post-processor that performs back mixing processing on the receptionside.

FIG. 12 is a diagram illustrating a configuration example of {Blendinginformation SEI message}.

FIG. 13 is a diagram illustrating a configuration example of{Blending_information( )}.

FIG. 14 is a diagram illustrating main content in the configurationexample in {Blending_information ( )}.

FIG. 15 is a diagram illustrating an example of a change in theinformation {Blending_information( )} that is inserted into each frame(picture frame) in moving image data Q at a high frame rate on whichmixing processing has been performed.

FIG. 16 is a diagram illustrating an example of a relation between themixing processing by the pre-processor and the back mixing processing bythe post-processor.

FIG. 17 is a diagram schematically illustrating an example of the mixingon the transmission side and the back mixing on the reception side.

FIG. 18 is a diagram illustrating a configuration example of a transportstream TS.

FIG. 19 is a block diagram illustrating a configuration example of atelevision receiver that has decoding ability with which moving imagedata at a high frame rate can be processed.

FIG. 20 is a block diagram illustrating a configuration example of atelevision receiver that has decoding ability with which moving imagedata at a normal frame rate can be processed.

FIG. 21 is a diagram illustrating another configuration example of thetransmission and reception system.

FIG. 22 is a flowchart illustrating an example of a control processingprocedure in a control unit (CPU) in a set top box.

FIG. 23 is a diagram illustrating an outline of processing performed bythe transmission apparatus, the set top box, and a display.

FIG. 24 is a diagram illustrating a case in which the display has afunction of the back mixing processing (mixing release processing) and acase in which the display does not have the function in a comparedmanner.

FIG. 25 is a diagram illustrating a configuration example of HFRblending infoframe.

FIG. 26 is a diagram illustrating content of main information in theconfiguration example of the HFR blending infoframe.

FIG. 27 is a diagram illustrating content of main information in theconfiguration example of the HFR blending infoframe.

FIG. 28 is a block diagram illustrating a configuration example of theset top box.

FIG. 29 is a block diagram illustrating a configuration example of thedisplay that deals with moving image data at a high frame rate.

FIG. 30 is a block diagram illustrating a configuration example of thedisplay that deals with moving image data at a normal frame rate.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment for implementing the present disclosure(hereinafter, referred to as an “embodiment”) will be described. Inaddition, the description will be given in the following order.

1. Embodiment 2. Modification Example 1. Embodiment <<Transmission andReception System>>

FIG. 1 illustrates a configuration example of a transmission andreception system 10 according to an embodiment. This transmission andreception system 10 has a transmission apparatus 100 and a televisionreceiver 200.

The transmission apparatus 100 transmits a transport stream TS as acontainer on a broadcasting wave. This transport stream TS includes abasic stream (basic video stream) obtained by processing moving imagedata at a high frame rate, that is, at 120 Hz or 240 Hz in thisembodiment and an extended stream (extended video stream). In thisembodiment, the basic stream and the extended stream have a NAL unitstructure.

Here, the basic stream is obtained as follows. That is, moving imagedata at a high frame rate after mixing processing is obtained byperforming processing of mixing, at a mixing rate independent for eachframe, image data in peripheral frames with image data in each frame ofthe moving image data at the high frame rate before the mixing.

At least image data in a frame corresponding to the normal frame rate,that is, 60 Hz in this embodiment in the image data in each frame thatforms the moving image data at the high frame rate after the mixingprocessing is brought into a state in which the image data is mixed withthe image data in the peripheral frames. The basic stream is obtained byencoding image data in a frame (basic frame) corresponding to the normalframe rate. In addition, the extended stream is obtained by encodingimage data in the residual frames (extended frames).

The basic stream includes coded image data in each frame at the normalframe rate as an access unit. In addition, the extended stream includesa coded image data in each extended frame at the high frame rate as anaccess unit.

(a) and (b) in FIG. 2 illustrates an example of the basic stream at 60Hz that is obtained by performing the mixing processing on moving imagedata at 120 Hz and an extended stream at +60 Hz. A frame pair is formedby one frame that forms the basic stream and one frame corresponding tothe following extended frame.

In (a) in FIG. 2, image data in the frame of the basic stream, which isa first frame, is in a state (mixed state) in which the image data ismixed with image data in the peripheral frames while image data in thefollowing frame of the extended stream is in a state (non-mixed state)in which the image data is not mixed with the image data in theperipheral frames, in each frame pair. In (b) in FIG. 2, image data inthe frame of the basic stream, which is a first frame, is in the state(mixed state) in which the image data is mixed with the image data inthe peripheral frames while image data in the following frame of theextended stream is also in the state (mixed state) in which the imagedata is mixed with the image data in the peripheral frames, in eachframe pair.

(a), (b), (c), and (d) in FIG. 3 illustrate an example of basic streamsat 60 Hz that are obtained by performing the mixing processing on movingimage data at 240 Hz and an extended stream added thereto. A frame pairis formed by four frames including one frame that forms the basic streamand the following three frames corresponding to the extended frames.

In (a) in FIG. 3, image data in a frame of the basic stream, which is afirst frame, is in the state (mixed state) in which the image data ismixed with image data in the peripheral frames while image data of allthe three following frames of the extended stream is in a state(non-mixed state) in which the image data is not mixed with the imagedata in the peripheral frames, in each frame pair.

In (b) in FIG. 3, the image data in the frame of the basic stream, whichis the first frame, is in the state (mixed state) in which the imagedata is mixed with the image data in the peripheral frames, in eachframe pair. In addition, image data in a following first frame of theextended stream is in the state (non-mixed state) in which the imagedata is not mixed with the image data in the peripheral frame, imagedata in a second frame is in the state (mixed state) in which the imagedata is mixed with the image data in the peripheral frames, and imagedata in a third frame is in the state (non-mixed state) in which theimage data is not mixed with the image data in the peripheral frames.

In (c) in FIG. 3, the image data in the frame of the basic stream, whichis the first frame, is in the state (mixed state) in which the imagedata is mixed with the image data in the peripheral frames, in eachframe pair. In addition, image data in a following first frame of theextended stream is in the state (mixed state) in which the image data ismixed with the image data in the peripheral frame, image data in asecond frame is in the state (non-mixed state) in which the image datais not mixed with the image data in the peripheral frames, and imagedata in a third frame is in the state (mixed state) in which the imagedata is mixed with the image data in the peripheral frames.

In (d) in FIG. 3, image data in a frame of the basic stream, which is afirst frame, is in the state (mixed state) in which the image data ismixed with image data in the peripheral frames while image data of allthe three following frames of the extended stream is also in a state(mixed state) in which the image data is mixed with the image data inthe peripheral frames, in each frame pair.

The information about the mixing rate in the corresponding frames isinserted into the basic stream and the extended stream in associationwith the image data in the respective frames. Here, the informationabout the mixing rate in each frame is respectively a set ofcoefficients corresponding to the number of taps of a filter used forthe mixing processing. In a case in which m-tap filter capable of mixingm frames is used, for example, the coefficient set of each frameincludes m coefficients. In this embodiment, a SEI NAL unit that has theinformation of the mixing rate (coefficient set) is inserted into thebasic stream and the extended stream. The reception side can recognizeat which rate the image data in each frame of the basic stream and theextended stream has mixed with peripheral image data, on the basis ofthe information about the mixing rate.

Returning to FIG. 1, the television receiver receives the aforementionedtransport stream TS sent on the broadcasting wave from the transmissionapparatus 100. In a case in which there is decoding ability with whichthe moving image data at the normal frame rate (60 Hz) can be processed,the reception apparatus 200 processes only the basic stream included inthe transport stream TS, obtains the moving image data at the normalframe rate, and reproduces the image. In this case, the televisionreceiver 200 performs decoding processing on the basic stream andobtains the image data in each frame at the normal frame rate.

Meanwhile, in a case in which there is decoding ability with which themoving image data at the high frame rate (120 Hz or 240 Hz) can beprocessed, the television receiver 200 processes both the basic streamand the extended stream included in the transport stream TS, obtains themoving image data at the high frame rate, and reproduces the image.

In this case, the television receiver 200 obtains the image data in eachframe at the normal frame rate by performing the decoding processing onthe basic stream, obtains the image data in each extended frame at thehigh frame rate by performing the decoding processing on the extendedstream, and then performs the back mixing processing by using image datain each frame at the normal frame rate and the image data in eachextended frame at the high frame rate on the basis of the information(coefficient set) about the mixing rate in each frame, thereby obtainingthe moving image data at the high frame rate that is similar to thatbefore the mixing processing.

FIG. 4 illustrates an outline of processing performed by thetransmission apparatus 100 and the television receiver 200. In addition,although an image sequence Q output from a pre-processor 102 of thetransmission apparatus 100 and an image sequence Q output from a decoder204 of the television receiver 200A are the same in a time seriesmanner, a case in which image quality in both the image sequences Q isdifferent is also included since the image sequences Q are made to passthrough a codec. Moving image data Va at a higher frame rate that isoutput from a camera (imaging apparatus) 81 is sent to an HFR processor82, and moving image data Vb at the high frame rate (120 Hz or 240 Hz)is obtained. This moving image data Vb is input as moving image data Pto the transmission apparatus 100.

In the transmission apparatus 100, the mixing processing is performed onthe image data in each frame that forms the moving image data P by thepre-processor 102, and moving image data Q after the mixing processingthat includes image data Qb in each frame at the normal frame rate andimage data Qe in each extended frame at the high frame rate is obtained.In the transmission apparatus 100, an encoder 103 performs encodingprocessing on the image data Qb and Qe, and a basic stream STb and anextended stream STe are obtained. The transmission apparatus 100transmits these streams STb and STe to the television receiver 200. Inaddition, the information about the mixing rate in the correspondingframes is inserted into these streams STb and STe in association withthe image data in the respective frames.

In a television receiver 200A that has decoding ability with which themoving image data at the high frame rate can be processed, the decoder204 performs decoding processing on the two streams STb and STe, and themoving image data Q, which includes the image data Qb in each frame atthe normal frame rate and the image data Qe in each extended frame atthe high frame rate, and on which the mixing processing has beenperformed, is obtained. Then, in the reception apparatus 200A, thepost-processor 205 performs the back mixing processing (mixing releaseprocessing) on the image data in each frame of the moving image data Qon the basis of the information about the mixing rate in each frame, andmoving image data R at the high frame rate (120 Hz or 240 Hz) that issimilar to the moving image data P on the transmission side is obtained.The moving image data R is used as moving image data for display withoutany processing performed thereon or by performing frame interpolation bya motion compensated frame insertion (MCFI) unit 206 to increase theframe rate.

Meanwhile, in a television receiver 200B that has decoding ability withwhich the image data at the normal frame rate can be processed, adecoder 204B performs the decoding processing on the stream STb, and theimage data Qb in each frame at the normal frame rate is obtained. Then,in the reception apparatus 200B, the moving image data including theimage data Qb in each frame at the normal frame rate is used as movingimage data for display without any processing performed or by performingframe interpolation by a motion compensated frame insertion (MCFI) unit206B to increase the frame rate.

FIG. 5 schematically illustrating an example of mixing (blending) on thetransmission side and back mixing (unblending) on the reception side.This example corresponds to the example in (a) in FIG. 2, a frame {n}and a frame {n+1} form a frame pair, and a frame {n+2} and a frame {n+3}form a frame pair. In addition, objects Oa and Ob are objects with nomotion while an object Oc is an object with motion in the exampleillustrated in the drawing.

The image data in the frame of the basic stream, which is the firstframe, is brought into a state (mixed state) in which the image data ismixed with image data in the peripheral frames, and image data in thefollowing frame in the extended stream is brought into a state(non-mixed state) in which the image data is not mixed with the imagedata in the peripheral frames in each frame pair by the mixingprocessing on the transmission side. In addition, the mixed state isreleased by the back mixing processing on the reception side.

FIG. 6 schematically illustrates another example of the mixing(blending) on the transmission side and the back mixing (unblending) onthe reception side. This example corresponds to the example in (a) inFIG. 3, and a frame {n} and frames {n+1} to {n+3} form frame pairs. Inaddition, objects Oa and Ob are objects with no motion while an objectOc is an object with motion in the example illustrated in the drawing.

The image data in the frame of the basic stream, which is the firstframe, is brought into a state (mixed state) in which the image data ismixed with image data in the peripheral frames, and image data in thefollowing three frames in the extended stream is brought into a state(non-mixed state) in which the image data is not mixed with the imagedata in the peripheral frames in each frame pair by the mixingprocessing on the transmission side. In addition, the mixed state isreleased by the back mixing processing on the reception side.

<<Configuration of Transmission Apparatus>>

FIG. 7 illustrates a configuration example of the transmission apparatus100. This transmission apparatus 100 has a control unit 101, thepre-processor 102, the encoder 103, a multiplexer 104, and atransmission unit 105. The control unit 101 controls operations of therespective parts in the transmission apparatus 100.

The pre-processor 102 inputs the moving image data P at the high framerate (120 Hz or 240 Hz) and outputs the image data Qb in each frame atthe normal frame rate (60 Hz) and the image data Qe in each extendedframe at the high frame rate.

Here, the pre-processor performs the processing of mixing, at the mixingrate independent for each frame, the image data in the peripheral frameswith the image data in each frame of the moving image data P at the highframe rate before the mixing processing and obtains the moving imagedata Q at the high frame rate after the mixing processing. Image data ineach frame corresponding to the normal frame rate (60 Hz) in the movingimage data Q is the image data Qb, and image data in each residual frameis the image data Qe. In this case, at least the image data Qb is in astate in which the image data Qb is mixed with the image data in theperipheral frames.

The pre-processor 102 includes a filter in a time direction that has twoor more taps, for example, and can perform the processing of mixing theimage data in the peripheral frames at the mixing rate independent foreach frame by changing the coefficient of each tap for each frame.

FIG. 8 illustrates a configuration example of the pre-processor 102.This preprocessor 102 includes a filter in the time direction that hasthree taps. This preprocessor 102 has delay elements 102 a and 102 bthat has a delay time of one frame period, coefficient devices 102 c,102 d, and 102 e, and an adder 102 f.

The moving image data P before the mixing processing is input to aserial circuit of the delay elements 102 a and 102 b. In this case, themoving image data P is sent in an order of the delay element 102 b andthe delay element 102 a. Image data P1, P2, and P3 in three frames isobtained as three tap outputs on the output side of the delay element102 a, the input side of the delay element 102 a (the output side of thedelay element 102 b), and the input side of the delay element 102 b. Theimage data P1, P2, and P3 is multiplied by coefficients a, b, and c bythe coefficient devices 102 c, 102 d, and 102 e, respectively. Thecoefficient set (a, b, and c) are switched for each frame. The outputsof the respective coefficient devices 102 c, 102 d, and 102 e are addedby the adder 102 f, and image data Q1, Q2, and Q3 in the respectiveframes of the moving image data Q after the mixing processing issequentially obtained from the adder 102 f.

FIG. 9 illustrates another configuration example of the pre-processor102. This preprocessor 102 includes a filter in the time direction thathas three taps. This preprocessor 102 is an example in which acoefficient set of a first system, a second system, and a third systemis sequentially repeated for each frame.

This pre-processor 102 has delay elements 102 a, 102 b that have a delaytime of one frame period, coefficient devices 102 c 1, 102 d 1, and 102e 1 in the first system, coefficient devices 102 c 2, 102 d 2, and 102 e2 in the second system, a coefficient devices 102 c 3, 102 d 3, and 102e 3 in the second system, an adder 102 f 1 in the first system, an adder102 f 2 in the second system, an adder 102 f 3 in the third system, anda frame output switching device (SW) 102 g.

The moving image data P before the mixing processing is input to aserial circuit of the delay elements 102 a and 102 b. In this case, themoving image data P is sent in the order of the delay element 102 b andthe delay element 102 a. Image data P1, P2, and P3 in three frames isobtained as three tap outputs on the output side of the delay element102 a, the input side of the delay element 102 a (the output side of thedelay element 102 b), and the input side of the delay element 102 b.

The output of the first system is obtained by multiplying the image dataP1, P2, and P3 by coefficients a1, b1, and c1 by the coefficient devices102 c 1, 102 d 1, and 102 e 1, respectively and then adding the resultsby the adder 102 f 1. In addition, the output of the second system isobtained by multiplying the image data P1, P2, and P3 by coefficientsa2, b2, and c2 by the coefficient devices 102 c 2, 102 d 2, and 102 e 2,respectively and then adding the results by the adder 102 f 2. Theoutput of the third system is obtained by multiplying the image data P1,P2, and P3 by coefficients a3, b3, and c3 by the coefficient devices 102c 3, 102 d 3, and 102 e 3, respectively and then adding the results bythe adder 102 f 3.

The outputs of the respective systems obtained by the adders 102 f 1,102 f 2, and 102 f 3 are selectively extracted for each frame by theframe output switching device 102 g, and the image data Q1, Q2, and Q3in the respective frames of the moving image data Q after the mixingprocessing is sequentially obtained from the frame output switchingdevice 102 g.

Here, when the frame {n} is at the position of P2, the frame outputswitching device 102 g selects a computation result of{P1*a1+P2*b1+P3*c1}. In addition, when the frame {n+1} is at theposition of P2, the frame output switching device 102 g selects acomputation result of {P1*a2+P2*b2+P3*c2}. In addition, when the frame{n+2} is at the position of P2, the frame output switching device 102 gselects a computation result of {P1*a3+P2*b3+P3*c3}.

The pre-processor 102 sequentially repeats the above processing. Inaddition, the pre-processor 102 includes a delay management functionsuch that a synchronization relation between the coefficient set and thecorresponding frames does not break.

FIG. 10 illustrates a configuration example of the post-processor 205that performs the back mixing processing on the reception side. Thisexample corresponds to the configuration example of the pre-processor102 illustrated in FIG. 8. This post-processor 205 includes a filterwith three taps. This post-processor 205 has delay elements 205 a and205 b that has a delay time of one frame period, coefficient devices 205c, 205 d, and 205 e, and an adder 205 f.

The moving image data Q before the back mixing processing is input to aserial circuit of the delay elements 205 a and 205 b. In this case, themoving image data Q is sent in the order of the delay element 205 b andthe delay element 205 a. Image data Q1, Q2, and Q3 in the three framesis obtained as three tap outputs on the output side of the delay element205 a, the input side of the delay element 205 a (the output side of thedelay element 205 b), and the input side of the delay element 205 b.

The image data Q1, Q2, and Q3 are multiplied by coefficients a′, b′, andc′ by the coefficient devices 205 c, 205 d, and 205 e, respectively. Thecoefficient set (a′, b′, and c′) is obtained to perform the back mixingprocessing (mixing release processing) from the coefficient set (a, b,and c) (see FIG. 8) at the time of the mixing processing that isprovided from the transmission side and is switched for each frame.Outputs of the respective coefficient devices 205 c, 205 d, and 205 eare added by the adder 205 f, and image data R1, R2, and R3 in therespective frames of the moving image data R after the back mixingprocessing is sequentially obtained from the adder 205 f.

FIG. 11 illustrates another configuration example of the post-processor205 that performs the back mixing processing on the reception side. Thisexample corresponds to the configuration example of the pre-processor102 illustrated in FIG. 9. This post-processor 205 includes a filterthat has three taps. This post-processor 205 is an example in which thecoefficient set of the first system, the second system, and the thirdsystem is sequentially repeated for each frame.

This post-processor 205 has delay elements 205 a, 205 b that have adelay time of one frame period, coefficient devices 205 c 1, 205 d 1,and 205 e 1 in the first system, coefficient devices 205 c 2, 205 d 2,and 205 e 2 in the second system, a coefficient devices 205 c 3, 205 d3, and 205 e 3 in the second system, an adder 205 f 1 in the firstsystem, an adder 205 f 2 in the second system, an adder 205 f 3 in thethird system, and a frame output switching device (SW) 205 g.

The moving image data P before the back mixing processing is input to aserial circuit of the delay elements 205 a and 205 b. In this case, themoving image data Q is sent in the order of the delay element 205 b andthe delay element 205 a. Image data Q1, Q2, and Q3 in the three framesis obtained as three tap outputs on the output side of the delay element205 a, the input side of the delay element 205 a (the output side of thedelay element 205 b), and the input side of the delay element 205 b.

In addition, the output of the first system is obtained by multiplyingthe image data Q1, Q2, and Q3 by coefficients a1, b1, and c1 by thecoefficient devices 205 c 1, 205 d 1, and 205 e 1, respectively and thenadding the results by the adder 205 f 1. In addition, the output of thesecond system is obtained by multiplying the image data Q1, Q2, and Q3by coefficients a2, b2, and c2 by the coefficient devices 205 c 2, 205 d2, and 205 e 2, respectively and then adding the results by the adder205 f 2. The output of the third system is obtained by multiplying theimage data Q1, Q2, and Q3 by coefficients a3, b3, and c3 by thecoefficient devices 205 c 3, 205 d 3, and 205 e 3, respectively and thenadding the results by the adder 205 f 3.

The coefficient sets (a1′, b1′, and c1′), (a2′, b2′, and c2′), and (a3′,b3′, c3′) are obtained to perform the back mixing processing (mixingrelease processing) from the coefficient sets (a1, b1, and c1), (a2, b2,and c2), and (a3, b3, and c3) (see FIG. 9) at the time of the mixingprocessing that are provided from the transmission side.

The outputs of the respective systems obtained by the adders 205 f 1,205 f 2, and 205 f 3 are selectively extracted for each frame by theframe output switching device 205 g, and the image data R1, R2, and R3in the respective frames of the moving image data R after the mixingprocessing is sequentially obtained from the frame output switchingdevice 205 g.

Here, when the frame {n} is at the position of Q2, the frame outputswitching device 205 g selects a computation result of{Q1*a1′+Q2*b1′+Q3*c1′}. In addition, when the frame {n+1} is at theposition of Q2, the frame output switching device 205 g selects acomputation result of {Q1*a2′+Q2*b2′+Q3*c2′}. In addition, when theframe {n+2} is at the position of Q2, the frame output switching device205 g selects a computation result of {Q1*a3′+Q2*b3′+Q3*c3′}.

The post-processor 205 sequentially repeats the above processing foreach frame. In addition, the post-processor 205 includes the delaymanagement function such that the synchronization relation between thecoefficient sets and the corresponding frames does not break.

Returning to FIG. 7, the encoder 103 performs encoding processing on theimage data Qb and Qe that is obtained by the pre-processor 102 andgenerates the basic stream STb and the extended stream STe. In thiscase, prediction encoding processing such as H.264/AVC or H.265/HEVC,for example is performed on the image data Qb and Qe.

The encoder 102 inserts the information about the mixing rate in thecorresponding frames into the basic stream STb and the extended streamSTe in association with the image data in the respective frames. Thereception side can recognize at which rate the image data in each frameof the basic stream and the extended stream has been mixed withperipheral image data, on the basis of the information about the mixingrate and can appropriately perform the back mixing processing (mixingrelease processing).

In this embodiment, a SEI NAL unit that has information about the mixingrate is inserted into the respective access units of the basic streamSTb and the extended stream STe. In this case, the encoder 103 inserts ablending information SEI message for new definition into a portion of“SEIs” in the access unit (AU).

(a) in FIG. 12 illustrates a structure example (Syntax) of{Blending_information SEI message}. {uuid_iso_iec_11578} has a UUIDvalue indicated by “ISO/IEC 11578:1996 AnnexA”. { } is inserted into thefield of {user_data_payload_byte}. (b) in FIG. 12 illustrates aconfiguration example (Syntax) of {Blending_information_SEI( )}, and{Blending_information ( )} including the information about the mixingrate is inserted therein. {userdata_id} is an identifier indicated by 16bits with no codes. 8-bit field of {Blending_information SEI_length}indicates a byte size of {Blending_information_SEI_length} at and afterthe field.

FIG. 13 illustrates a structure example (Syntax) of{Blending_information( )}, and FIG. 14 illustrates content (Semantics)of main information in the structure example. 3-bit field {frame_rate}represents a frame rate. For example, “3” represents 120 Hz, and “5”represents 240 Hz.

1-bit field of {blending_flag} represents whether or not the mixingprocessing with the peripheral image data is to be applied. For example,“0” represents that no application is to be made, and “1” representsthat application is to be made. 2-bit field of {temporal_filter_taps} isconfiguration information of the filter used for performing the mixingprocessing and represents the number of frames used (the number oftaps). For example, “0” represents two frames (two taps), and “1”represents three frames (three taps).

1-bit field of {first_frame_in_frame-pair_flag} indicates whether theframe is the first frame of the frame pair, that is, whether the frameis a frame corresponding to the temporal display position at the normalframe rate. For example, “0” represents that the frame is a frame otherthan the first frame, and “1” represents that the frame is the firstframe. In addition, the frame pair includes the one frame that forms thebasic stream and the following predetermined number of extended framesas described above (see FIGS. 2 and 3).

3-bit field of {blending_refresh_distance (BR distance)} represents thenumber of frames (temporal distance) until temporal linkage of themixing processing is refreshed without using the image data in thetemporarily previous frame. For example, “0” represents one frame, “1”represents two frames, and “2” represents three frames. 1-bit field of{refresh_flag} indicates whether or not the frame refreshes theaforementioned temporal linkage of the mixing processing. For example,“0” represents that the mixing refresh is not performed for the frame,and “1” represents that the mixing refresh is performed for the frame.

8-bit field of {blend_coefficient} is repeatedly present to correspondto the number of frames (the number of taps) represented by theaforementioned field of {temporal_filter_taps}. This field represents acoefficient (blend ratio) by which the image data in each frame ismultiplied. For example, “0×0” represents {0}, “0×1” represents {1/5},“0×2” represents {1/4}, “0×3” represents {1/3}, “0×4” represents {1/2},“0×5” represents {2/3}, “0×6” represents {3/4}, “0×7” represents {4/5},and “0×8” represents {1}.

FIG. 15 illustrates an example of a change in the information of{Blending_information( )} to be inserted into each frame (picture frame)in the moving image data Q at the high frame rate, on which the mixingprocessing has been performed. The example illustrated in the drawingillustrates a case in which {temporal_filter_taps} is “1” and the numberof frames (the number of taps) is {3}.

For example, the frame {n} included in the basic stream represents that{refresh_flag} is “1” and the mixing refresh is to be performed for theframe, and also {bleding_refresh_distance (BR distance)} is “3” and thenumber of frames until the frame for which the next mixing refresh is tobe performed is four frames. In addition, the fact that the frame of thecenter coefficient in the three frames (three taps) is a current frame,and the frames are {(=1), 6 (=3/4), 2 (=1/4)} from the old one isrepresented.

In addition, the frame {n+1} included in the extended frame representsthat {refresh_flag} is “0” and the mixing refresh is not to be performedfor the frame, and also {blending_refresh_distance (BR distance)} is “2”and the number of frames until the frame for which the next mixing freshis performed is three frames. In addition, the fact that the frame ofthe center coefficient in the three frames (three taps) is a currentframe, and the frames are {(=0), 8 (=1), and 0 (=0)} from the old one isrepresented.

In addition, the frame {n+2} included in the basic frame, for example,represents that {refresh_flag} is “0” and the mixing fresh is not to beperformed for the frame, and also {blending_refresh_distance (BRdistance)} is “1” and the number of frames until the frame for which thenext mixing refresh is performed is two frames. In addition, the factthat the frame of the center coefficient in the three frames (threetaps) is the current frame, and the frames are {2 (=1/4), 4 (=1/2), and2 (=1/4)} from the old one is represented.

In addition, the frame {n+3} included in the extended frame representsthat {refresh_flag} is “0” and the mixing refresh is not to be performedfor the frame, and also {blending_refresh_distance (BR distance)} is “0”and the number of frames until the frame for which the next mixing freshis performed is one frame. In addition, the fact that the frame of thecenter coefficient in the three frames (three taps) is a current frame,and the frames are {(=0), 8 (=1), and 0 (=0)} from the old one isrepresented.

For example, the frame {n+4} included in the basic stream representsthat {refresh_flag} is “1” and the mixing refresh is to be performed forthe frame, and also {bleding_refresh_distance (BR distance)} is “3” andthe number of frames until the frame for which the next mixing refreshis to be performed is four frames. In addition, the fact that the frameof the center coefficient in the three frames (three taps) is a currentframe, and the frames are {0 (=1), 6 (=3/4), 2 (=1/4)} from the old oneis represented.

FIG. 16 corresponds to the example in FIG. 15 and illustrates an exampleof a relation between the mixing processing performed by thepre-processor 102 and the back mixing processing performed by thepost-processor 205. The coefficient (mixing ratio) used in the backmixing processing performed by the post-processor 205 is obtained on thebasis of a coefficient (mixing ratio) used in the latest processing.

In addition, the aforementioned case of the example illustrated in FIG.5 can be realized by setting {temporal_filter_taps} to “1 (threeframes)”, setting the coefficients of the three frames (three taps) inthe frame {n} to {(=0), 4 (=1/2), and 4=(=1/2)}, setting coefficients ofthe three frames (three taps) in the frame {n+1} to {(=0), 8 (=1), and 0(=0)}, setting the coefficients of the three frames (three taps) in theframe {n+2} to {0 (=0), 4 (=1/2), 4 (=1/2)}, setting the coefficients ofthe three frames (three taps) in the frame {n+3} to {0 (=0), 8 (=1), and0 (=0)}, and repeating this thereafter.

In addition, in a case in which mixing with the next frame is performedin a progressive manner, that is, in a case in which coefficient linkagebetween the frames in the mixing processing continues over two or moreframes, setting is made such that only the frame with {refresh_flag} of“1” has {first_frame_in_frame-pair_flag} of “1”. The post-processor 205performs the back mixing processing from the frame with {refresh_flag}of “1”. The back mixing processing is performed from the previous frameof the frame with {refresh_flag} of “1”, that is, in the order of theframe {n+2}, the frame {n+1}, and the frame {n} while tracking back fromthe frame {n+3} in FIG. 16. At this time, the target maximum number offrames is represented by {blending refresh distance (BR distance)}, andthe value of the frame number {n} is four frames in the example in FIG.16. FIG. 17 schematically illustrates an example of the mixing(blending) on the transmission side and the back mixing (unblending) onthe reception side.

Returning to FIG. 7, the multiplexer 104 packetized-elementary-stream(PES) packetizes the basic stream STb and the extended stream STe thatare generated by the encode 103 and further transport-packetizes andmultiplexes the streams, thereby obtaining the transport stream TS as amultiplexed stream.

FIG. 18 illustrates a configuration example of the transport stream TS.This transport stream TS includes two video streams, namely the basicstream (base stream) STb and the extended stream (enhanced stream) STe.That is, a PES packet {video PES1} of the basic stream STb is present,and also a PES packet {video PES2} of the extended stream STe is presentin this configuration example.

The blending information SEI message (see FIG. 13) is inserted intoencoded image data of each picture that forms a container of the PESpacket {video PES1} and the PES packet {video PES2}.

In addition, the transport stream TS includes a program map table (PMT)as one piece of program specific information (PSI). The PSI isinformation describing which programs the respective elementary streamsincluded in the transport stream belong to.

A program loop (Programloop) that describes information related to theentire programs is present in the PMT. In addition, an elementary streamloop that has information related to the respective video streams ispresent in the PMT. In this configuration example, a video elementarystream loop {video ES1 loop} corresponding to the basic stream ispresent, and a video elementary stream loop {video ES2 loop}corresponding to the extended stream is present.

Information such as a stream type and a packet identifier (PID) isarranged to correspond to the basic stream (video PES1), and also adescriptor that describes the information related to the video stream isalso arranged in {video ES1 loop}. This stream type is “0×24” indicatingthe basic stream in a case of HEVC coding.

In addition, information such as a stream type and a packet identifier(PID) is arranged to correspond to the extended stream (video PES2), andalso, a descriptor that describes the information related to the videostream is also arranged in {video ES2 loop}. This stream type is “0×25”indicating the extended stream.

In addition, although the example illustrated in the drawing illustratesa case in which

HEVC coding is performed, transmission of signaling information by theblending information SEI message can similarly be applied to anothercodec. In a case of another codec, a different description is insertedinto the PMT.

Returning to FIG. 7, the transmission unit 105 modulates the transportstream TS by a modulation scheme that is suitable for broadcasting, suchas QPSK/OFDM, and transmits an RF modulation signal to a transmissionantenna.

Operations of the transmission apparatus 100 illustrated in FIG. 7 willbe briefly described. Moving image data P at the high frame rate (120 Hzor 240 Hz) is input to the pre-processor 102. In this pre-processor 102,the mixing processing is performed on the moving image data P, and theimage data Qb in each frame at the normal frame rate (60 Hz) and theimage data Qe in each extended frame at the high frame rate areobtained.

In this case, the processing of mixing, at the mixing rate independentfor each frame, the image data in the peripheral frames with the imagedata in each frame of the moving image data P at the high frame ratebefore the mixing processing is performed, and the moving image data Qat the high frame rate after the mixing processing is obtained in thepre-processor 102. In this moving image data Q, image data in each framecorresponding to the normal frame rate (60 Hz) is regarded as the imagedata Qb, and image data in the residual respective frames is regarded asthe image data Qe. In this case, at least the image data Qb is broughtinto a state in which the image data Qb is mixed with the image data inthe peripheral frames.

The image data Qb and Qe obtained by the pre-processor 102 is suppliedto the encoder 103. In the encoder 103, encoding processing is performedon the image data Qb and Qe, and the basic stream STb and the extendedstream STe are generated. In this encoder 103, the information about themixing ratio in the mixing processing is inserted onto the basic streamSTb and the extended stream STe.

The encoder 103 inserts the information about the mixing rate in thecorresponding frames into the basic stream STb and the extended streamSTe in association with the image data in the respective frames. Thereception side can recognize at which rate the image data in each frameof the basic stream and the extended stream has been mixed withperipheral image data, on the basis of the information about the mixingrate and can appropriately perform the back mixing processing (mixingrelease processing).

The basic stream STb and the extended stream STe generated by theencoder 103 is supplied to the multiplexer 104. The multiplexer 104 PESpacketizes the basic stream STb and the extended stream STe and furthertransport-packetizes and multiplexes the streams, thereby obtaining thetransport stream TS as a multiplexed stream.

The transport stream TS generated by the multiplexer 104 is sent to thetransmission unit 105. The transmission unit 105 modulates thistransport stream TS by a modulation scheme that is suitable forbroadcasting, such as QPSK/OFDM, for example, and transmits an RFmodulation signal from the transmission antenna.

<<Configuration of Television Receiver>>

FIG. 19 illustrates a configuration example of the television receiver200A that has decoding ability with which moving image data at the highframe rate (120 Hz or 240 Hz) can be processed. This television receiver200A has a control unit 201, a reception unit 202, a demultiplexer 203,a decoder 204, the post-processor 205, a motion compensated frameinsertion (MCFI) unit 206, and a panel display unit 207.

The control unit 201 controls operations of the respective parts in thetelevision receiver 200A. The reception unit 202 demodulates the RFmodulation signal received by a reception antenna and acquires thetransport stream TS. The demultiplexer 203 extracts the basic stream STband the extended stream STe by PID filtering from the transport streamTS and supplies the basic stream STb and the extended stream STe to thedecoder 204.

The decoder 204 performs decoding processing on the basic stream STb toobtain the image data Qb in each frame at the normal frame rate andperforms decoding processing on the extended stream STe to obtain theimage data Qe in each extended frame at the high frame rate. In thismanner, the moving image data Q at the high frame rate, on which themixing processing of the image data Qb and the image data Qe has beenperformed, is obtained.

In addition, the decoder 204 extracts a parameter set and SEI that havebeen inserted into the respective access units that form the basicstream STb and the extended stream STe and sends the parameter set andthe SEI to the control unit 201. In this case, the blending informationSEI (see FIG. 13) that has information about the mixing rate isextracted. In this manner, the control unit 201 can recognize at whichrate the image data in each frame of the moving image data Q at the highframe rate after the mixing processing has been mixed with theperipheral image data, obtain coefficients for the back mixingprocessing, and satisfactorily control the post-processor 205, whichwill be described later.

The post-processor 205 performs the back mixing processing (mixingrelease processing) on the moving image data Q at the high frame rate,which has been obtained by the decoder 204, under the control by thecontrol unit 201 and obtains mixing-released moving image data R at thehigh frame rate. The post-processor 205 includes a filter that has twoor more taps, for example, and can release the mixing by changing thecoefficient of each tap for each frame and mixing the image data in theperipheral frames at the mixing rate independent for each frame (seeFIGS. 10, 11, and 16). In this case, the control unit 201 calculates thecoefficient set for each frame on the basis of the coefficient set atthe time of the mixing processing, which has been inserted into eachframe of the moving image data Q, and uses the coefficient set asdescribed above.

The MCFI unit 206 performs frame interpolation processing of motioncompensation on the moving image data R at the high frame rate, whichhas been obtained by the post-processor 205, and obtains the movingimage data at a further increased frame rate. In addition, there is alsoa case in which this MCFI unit 206 is not provided. The panel displayunit 207 performs image display based on the moving image data R at thehigh frame rate that has been obtained by the post-processor 205 or themoving image data at a frame rate increased by the MCFI unit 206.

Operations of the television receiver 200A illustrated in FIG. 19 willbe briefly described. The reception unit 202 demodulates the RFmodulation signal received by the reception antenna and acquires thetransport stream TS. This transport stream TS is sent to thedemultiplexer 203. The demultiplexer 203 extracts the basic stream STband the extended stream STe from the transport stream TS by the PIDfiltering and supplies the basic stream STb and the extended stream STeto the decoder 204.

The decoder 204 performs the decoding processing on the basic stream STband the extended stream STe and obtains the moving image data Q at thehigh frame rate after the mixing processing that includes the image dataQb in each frame at the normal frame rate and the image data Qe in eachextended frame at the high frame rate. The moving image data Q issupplied to the post-processor 205.

In addition, the decoder 204 extracts a parameter set and SEI insertedinto each access unit that forms the basic stream STb and the extendedstream STe and sends the parameter set and the SEI to the control unit201. In this case, the blending information SEI (see FIG. 13) that hasinformation about the mixing rate is also extracted. The control unit201 computes the coefficients for the back mixing processing on thebasis of the information about the mixing rate (coefficient set) in eachframe.

The post-processor 205 performs the back mixing processing (mixingrelease processing) on the moving image data Q at the high frame rateunder control by the control unit 201 and obtains the mixing-releasedmoving image data R at the high frame rate. In this case, thecoefficients for the back mixing processing are provided from thecontrol unit 201 to the post-processor 205.

The mixing-released moving image data R at the high frame rate that hasbeen obtained by the host processor 205 or the moving image data at theframe rate further increased by the MCFI unit 206 is supplied to thepanel display unit 207, and the panel display unit 207 performs imagedisplay based on the moving image data.

FIG. 20 illustrates a configuration example of a television receiver208B that has decoding ability with which the moving image data at thenormal frame rate (60 Hz) can be processed. This television receiver200B has a control unit 201B, a reception unit 202B, a demultiplexer203B, a decoder 204B, an MCFI unit 206B, and a panel display unit 207B.

The control unit 201B controls operations of the respective parts in thetelevision receiver 200B. The receiver unit 202B demodulates the RFmodulation signal received by the reception antenna and acquires thetransport stream TS. The demultiplexer 203B extracts the basic streamSTb from the transport stream Ts by the PID filtering and supplies thebasic stream STb to the decoder 204B. The decoder 204B performs thedecoding processing on the basic stream STb and obtains the moving imagedata at the normal frame rate that includes the image data Qb in eachframe at the normal frame rate.

The MCFI unit 206B performs frame interpolation processing of motioncompensation on the moving image data at this normal frame rate andobtains the moving image data at a further increased frame rate. Inaddition, there is also a case in which this MCFI unit 206 is notprovided. The panel display unit 207B performs image display based onthe moving image data R at the normal frame rate that has been obtainedby the post-processor 205 or the moving image data at a frame rateincreased by the MCFI unit 206.

Operations of the television receiver 200B illustrated in FIG. 20 willbe briefly described. The reception unit 202B demodulates the RFmodulation signal received by the reception antenna and acquires thetransport stream TS. This transport stream TS is sent to thedemultiplexer 203B. The demultiplexer 203B extracts the basic stream STbfrom the transport stream TS by the PID filtering and supplies the basicstream STb to the decoder 204.

The decoder 204B performs the decoding processing on the basic streamSTb and obtains the moving image data at the normal frame rate thatincludes the image data Qb in each frame at the normal frame rate. Themoving image data at the normal frame rate obtained by the decoder 204Bor the moving image data at the frame rate further increased by the MCFIunit 206B is supplied to the panel display unit t207B, and the paneldisplay unit 207B performs image display based on the moving image data.

As described above, at least the image data in the frame correspondingto the normal frame rate in the image data in each frame that forms themoving image data at the high frame rate is mixed with the image data inthe peripheral frames and is brought into a state in which the shutteraperture ratio is increased, and basic stream STb obtained by encodingthe image data in the frame corresponding to the normal frame rate istransmitted in the transmission and reception system 10 illustrated inFIG. 1.

Therefore, in a case of the television receiver 200B that has decodingability with which the moving image data at the normal frame rate can beprocessed, it is possible to display a smooth image as a moving image byprocessing the basic stream and obtaining the moving image data at thenormal frame rate and to avoid occurrence of the problem in imagequality in the frame interpolation processing due to low loadcomputation in the display processing.

In addition, the extended stream STe obtained by encoding the image datain the residual frames is obtained along with the basic stream STb, inthe transmission and reception system 10 illustrated in FIG. 1, and theinformation (coefficient set) about the mixing rate in the correspondingframes is inserted into the basic stream STb and the extended stream STein association with the image data in the respective frames and is thentransmitted. Therefore, in a case of a receiver that has decodingability with which the moving image data at the high frame rate can beprocessed, it is possible to easily obtain the mixing-released movingimage data at the high frame rate on the basis of the information aboutthe mixing rate in each frame and to satisfactorily display the movingimage at the high frame rate.

2. Modification Example

In addition, although the example of the transmission and receptionsystem 10 that includes the transmission apparatus 100 and thetelevision receiver 200 has been described in the aforementionedembodiment, the configuration of the transmission and reception systemto which the present technology can be applied is not limited thereto. Acase is also considered in which a portion of the television receiver200 includes a set top box and a display that are connected by a digitalinterface such as a high-definition multimedia interface (HDMI), forexample. Note that {HDMI} is a registered trademark.

FIG. 21 illustrates a configuration example of a transmission andreception system 10A. This transmission and reception system 10A has aconfiguration having the transmission apparatus 100, a set top box (STB)200-1, and a display 200-2. The set top box (STB) 200-1 and the display200-2 are connected by the HDMI.

Since the transmission apparatus 100 is the same as the transmissionapparatus 100 in the transmission and reception system 10 illustrated inFIG. 1, the description thereof will be omitted. A set top box 200-1receives the transport stream TS sent on the broadcasting wave from thetransmission apparatus 100.

In a case in which the display 200-2 can deal with the moving image dataat the high frame rate (120 Hz or 240 Hz), the set top box 200-1processes both the basic stream STb and the extended stream STe includedin the transport stream TS and obtains the moving image data Q at thehigh frame rate, on which the mixing processing has been performed.

In a case in which the display 200-2 has the function of the back mixingprocessing (mixing release processing), the set top box 200-1 sends themoving image data Q at the high frame rate, on which the mixingprocessing has been performed, and the information (coefficient set)about the mixing rate in each frame to the display 200-2 via the HDMItransfer path.

In addition, in a case in which the display 200-2 does not have thefunction of the back mixing processing (mixing release processing), theset top box 200-1 performs the back mixing processing (mixing releaseprocessing) on the moving image data Q at the high frame rate, on whichthe mixing processing has been performed, on the basis of theinformation (coefficient set) about the mixing rate in each frame andobtains the mixing-released moving image data R at the high frame rate.Then, the set top box 200-1 sends the moving image data R at the highframe rate to the display 200-2 via the HDMI transfer path.

Meanwhile, in a case in which the display 200-2 can deal with the movingimage data at the normal frame rate (60 Hz), the set top box 200-1processes only the basic stream STb included in the transport stream TSand obtains the moving image data at the normal frame rate that includesthe image data Qb in each frame at the normal frame rate. Then, the settop box 200-1 sends the moving image data at the normal frame rate tothe display 200-2 via the HDMI transfer path.

The set top box 200-1 that serves as a source device acquires EDID fromthe display 200-2 that serves as a sink device, determines whether ornot the display 200-2 can deal with the moving image data at the highframe rate (120 Hz or 240 Hz), and further determines whether or not thedisplay 200-2 has the function of the back mixing processing (mixingrelease processing).

The flowchart in FIG. 22 illustrates an example of a control processingprocedure in a control unit (CPU) of the set top box 200-1. First, thecontrol unit starts control processing in Step ST1. Next, the controlunit reads and checks EDID from the display 200-2 in Step ST2. Then, thecontrol unit determines whether or not the display 200-2 can deal withthe moving image data at the high frame rate (120 Hz or 240 Hz) in StepST3.

When the display 200-2 is not capable of dealing with the moving imagedata, the control unit decodes only the basic stream STb and transmitsthe moving image data at the normal frame rate that includes the imagedata Qb in each frame at the normal frame rate to the set top box 200-1in Step ST4. The control unit completes the control processing in StepST5 after this processing in Step ST4.

In addition, when the display 200-2 can deal with the moving image dataat the high frame rate in Step ST3, the control unit decodes the basicstream STb and the extended stream STe in Step ST6.

Next, the control unit determines whether or not the display 200-2 hasthe function of the back mixing processing (mixing release processing)in Step ST7. When the display 200-2 does not have the function of theback mixing processing, the control unit decides that the back mixingprocessing is performed on the side of the set top box 200-1 andtransmits the mixing-released moving image data R at the high frame rateto the set top box 200-1 in Step ST8. The control unit completes thecontrol processing in Step ST5 after this processing in Step ST8.

In addition, when the display 200-2 has the function of the back mixingprocessing in

Step ST7, the control unit decides that the back mixing processing isperformed on the side of the display 200-2 and sends the moving imagedata Q at the high frame rate, on which the mixing processing has beenperformed, and the information (coefficient set) about the mixing ratein each frame to the display 200-2 via the HDMI transfer path in StepST9. The control unit completes the control processing in Step ST5 afterthis processing in Step ST9.

FIG. 23 illustrates an outline of processing performed by thetransmission apparatus 100, the set top box 200-1, and the display200-2. In addition, although the image sequence Q output from thepre-processor 102 of the transmission apparatus 100 and the imagesequence Q output from the decoder 204 of the set top box 200-1 are thesame in the time-series manner, a case in which image quality isdifferent in both the image sequences Q is also included since the imagesequences Q are made to pass through a codec. Since the transmissionapparatus is similar to that described above in FIG. 4, the descriptionthereof will be omitted here. In a case in which a display 200-2Acapable of dealing with the moving image data at the high frame rate(120 Hz or 240 Hz) is connected to the set top box 200-1, the decoder204 performs the decoding processing on the two streams STb and STe andobtains the moving image data Q, which includes the image data Qb ineach frame at the normal frame rate and the image data Qe in eachextended frame at the high frame rate, on which the mixing processinghas been performed.

In addition, in a case in which the display 200-2A has the function ofthe back mixing processing (mixing release processing), the set top box200-1 transmits the moving image data Q and the information (coefficientset) about the mixing rate in each frame to the display 200-2A via theHDMI transfer path. The example illustrated in the drawing illustrates acase in which the display 200-2A includes the post-processor 205 and thedisplay 200-2A has the function of the back mixing processing (mixingrelease processing). In addition, (a) in FIG. 24 illustrates a state inthis case.

In addition, in a case in which the display 200-2A does not have thefunction of the back mixing processing (mixing release processing), theset top box 200-1 performs the back mixing processing (mixing releaseprocessing) on the moving image data Q by the post-processor 200-5 thatthe set top box 200-1 itself has and obtains the mixing-released movingimage data R at the high frame rate. Then, the set top box 200-1transmits this moving image data R to the display 200-2A via the HDMItransfer path. (b) in FIG. 24 illustrates a state in this case.

In addition, in a case in which a display 200-2B capable of dealing withthe moving image data at the normal frame rate (60 Hz) is connected tothe set top box 200-1, the decoder 204 performs the decoding processingon the stream ST and obtains the moving image data at the normal framerate that includes the image data Qb in each frame at the normal framerate. Then, the set top box 200-1 transmits the moving image data to thedisplay 200-2B via the HDMI transfer path.

The set top box 200-1 transmits the moving image data Q, on which themixing processing has been performed, and the information (coefficientset) about the mixing rate in each frame to the display 200-2A that candeal with the moving image data at the high frame rate (120 Hz or 240Hz) and has the function of the back mixing processing (mixing releaseprocessing) via the HDMI transfer path as described above.

In this case, the information (coefficient set) about the mixing rate isinserted into and transmitted with the blanking period of the movingimage data Q, for example. Here, an HFR blending infoframe newly definedis used.

FIG. 25 illustrates a structure example (syntax) of the HFR blendinginfoframe, and FIGS. 26 and 27 illustrate content (semantics) of maininformation in the structure example. The first 3 bytes of thisinfoframe correspond to a header portion, and information about aninfoframe type, a version number, a byte length of data byte is arrangedtherein.

3-bit information of {frame rate} is arranged from the seventh to fifthbits of data byte 1 (Date Byte 1). The 3-bit information represents aframe rate. For example, “3” represents 120 Hz, and “5” represents 240Hz. In addition, 1-bit information of {blending_flag} is arranged at thefourth bit of the data byte 1 (Date Byte 1). The 1-bit informationrepresents whether or not the mixing processing with the peripheralimage data is to be applied. For example, “0” represents the mixingprocessing is not to be applied, and “1” represents the mixingprocessing is to be applied.

In addition, 2-bit information of {temporal_filter_taps} is arrangedfrom the third to second bits of the data byte 1 (Date Byte 1). This2-bit information is configuration information of the filter used forperforming the mixing processing and represents the number of framesused (the number of taps). For example, “0” represents two frames (twotaps), and “1” represents three frames (three taps).

In addition, 1-bit information of {first_frame_in_frame-pair_flag (FF)}is arranged at the first bit of the data byte 1 (Date Byte 1). This1-bit information represents whether or not the frame is the first frameof the frame pair (frame-pair), that is, whether or not the framecorresponds to the normal frame rate). For example, “0” represents thatthe frame is a frame other than the first frame, and “1” represents thatthe frame is the first frame.

In addition, 1-bit information of {Synchronized Frame (SF)} is arrangedat the 0-th bit of the data byte 1 (Date Byte 1). This 1-bit informationrepresents whether or not it is necessary to perform synchronizationprocessing with the next video frame. For example, “0” represents thatit is not necessary to perform the synchronization processing with thenext video frame, and “1” represents that it is necessary to perform thesynchronization processing with the next video frame.

In addition, 3-bit information of {blending_refresh_distance (BRdistance)} is arranged from the seventh to fifth bits of data byte 2(Date Byte 2). This 3-bit in-formation represents the number of frames(temporal distance) until the temporal linkage of the mixing processingis refreshed without using image data in a temporarily previous frame.For example, “0” represents one frame, “1” represents two frames, and“2” represents three frames.

In addition, 1-bit information of {refresh flag} is arranged at thefourth bit of the data byte 2 (Date Byte 2). This 1-bit informationrepresents whether or not the frame is a frame for which theaforementioned temporal linkage of the mixing processing is refreshed.For example, “0” represents that the frame is a frame for which themixing refresh is not performed, and “1” represents that the frame is aframe for which the mixing refresh is performed.

In addition, blending ratios (coefficients) for the first to fifthpicture frames are arranged from data byte 3 (Date Byte 3) to data byte7 (Date Byte 7). Here, to which order of data byte the blending ratiosare arranged correspond to the number of frames (the number of taps)represented by a field of {temporal filter taps}. For example, “0×0”represents {0}, “0×1” represents {1/5}, “0×2” represents {1/4}, “0×3”represents {1/3}, “0×4” represents {1/2}, “0×5” represents {2/3}, “0×6”represents {3/4}, “0×7” represents {4/5}, and “0×8” represents {1}.

In the set top box 200-1, information included in the blendinginformation SEI message (see FIG. 13) is used without any processingperformed thereon as in-formation other than Synchronized Frame, thatis, information of {frame_rate}, {blending_flag},{temporal_filter_taps}, {first_frame_in_frame-pair_flag},{blending_refresh_distance (BR distance)}, and {refresh_flag} to beincluded in the HFR blending infoframe.

FIG. 28 illustrates a configuration example of the set top box 200-1. InFIG. 28, the same reference numerals are given to the partscorresponding to those in FIG. 19. This set top box 200-1 has a controlunit 201-1, the reception unit 202, the demultiplexer 203, the decoder204, the post-processor 205, and an HDMI transmission unit 208.

The control unit 201-1 controls operations of the respective parts inthe set top box 200-1. The reception unit 202 demodulates the RFmodulation signal received by the reception antenna, acquires thetransport stream TS, and sends the transport stream TS to thedemultiplexer 203.

The demultiplexer 203 extracts both the basic stream STb and theextended stream STe or extracts the basic stream STb by the PIDfiltering depending on whether or not the display 200-2 can deal withthe moving image data at the high frame rate (120 Hz or 240 Hz).

When the demultiplexer 203 extracts the basic stream STb and theextended stream STe, the decoder 204 performs the decoding processing onthe basic stream STb to obtain the image data Qb in each frame at thenormal frame rate and performs the decoding processing on the extendedstream STe to obtain the image data Qe in each extended frame at thehigh frame rate. That is, the moving image data Q at the high framerate, on which the mixing processing of the image data Qb and the imagedata Qe has been performed, is obtained.

In addition, the decoder 204 extracts the parameter set and the SEI thathave been inserted into each access unit that forms the basic stream STband the extended stream STe and sends the parameter set and the SEI tothe control unit 201-1. In this case, the blending information SEI (seeFIG. 13) that has the information about the mixing rate is alsoextracted.

In this manner, the control unit 201-1 can recognize at which rate theimage data in each frame of the moving image data Q at the high framerate after the mixing processing has been mixed with the peripheralimage data, obtain the coefficients for the back mixing processing, andsatisfactorily control the post-processor 205, which will be describedlater. In addition, the control unit 201-1 can obtain various kinds ofinformation arranged in the HFR blending infoframe from the blendinginformation SELL when the HFR blending infoframe is transmitted to thedisplay 200-2.

In addition, when the demultiplexer 203 extracts only the basic streamSTb, the decoding processing is performed on the basic stream STb, andthe moving image data at the normal frame rate that includes the imagedata Qb in each frame at the normal frame rate is obtained.

In a case in which the display 200-2 can deal with the moving image dataat the high frame rate and does not have the function of the back mixingprocessing, the post-processor 205 performs the back mixing processing(mixing release processing) on the moving image data Q at the high framerate that has been obtained by the decoder 204 and obtains themixing-released moving image data R at the high frame rate.

The HDMI transmission unit 208 transmits non-compressed moving imagedata to the display 200-2 via the HDMI transfer path throughcommunication in accordance with the HDMI. Here, in a case in which thedisplay 200-2 can deal with the moving image data at the high frame rateand does not have the function of the back mixing processing, themixing-released moving image data R at the high frame rate that has beenobtained by the post-processor 205 is transmitted to the display 200-2via the HDMI transfer path.

In a case in which the display 200-2 can deal with the moving image dataat the high frame rate and has the function of the back mixingprocessing, the moving image data Q at the high frame rate, which hasbeen obtained by the decoder 204, on which the mixing processing hasbeen performed, is transmitted to the display 200-2 via the HDMItransfer path. In this case, since the back mixing processing isperformed on the side of the display 200-2, the HFR blending infoframe(see FIG. 25) including the information about the mixing rate isinserted into and transmitted with the blanking period of the image datain each frame that forms the moving image data Q.

In addition, in a case in which the display 200-2 can deal with themoving image data at the normal frame rate, the moving image data at thenormal frame rate that includes the image data Qb in each frame at thenormal frame rate that has been obtained by the decoder 204 istransmitted to the display 200-2 via the HDMI transfer path.

FIG. 29 illustrates a configuration example of the display 200-2A thatdeals with the moving image data at the high frame rate. In FIG. 29, thesame reference numerals are given to the parts corresponding to those inFIG. 19. The display 200-2A has a control unit 201-2A, an HDMI receptionunit 209, the post-processor 205, the MCFI unit 206, and the paneldisplay unit 207. In addition, there is also a case in which thepost-processor 205 is not provided.

The control unit 201-2A controls operations of the respective parts inthe display 200-2A. The HDMI reception unit 209 receives non-compressedmoving image data at the high frame rate from the set top box 200-1 viathe HDMI transfer path through the communication in accordance with theHDMI. Here, in a case in which the post-processor 205 is not present,the mixing-released moving image data R at the high frame rate isreceived.

Meanwhile, in a case in which the post-processor 205 is present, themoving image data Q at the high frame rate, on which the mixingprocessing has been performed, is received. In this case, the HFRblending infoframe (see FIG. 25) which has been inserted into theblanking period of the moving image data Q is extracted and sent to thecontrol unit 201-2A. In this manner, the control unit 201-2A canrecognize at which rate the image data in each frame of the moving imagedata Q at the high frame rate after the mixing processing has been mixedwith the peripheral image data, obtain the coefficients for the backmixing processing, and satisfactorily control the post-processor 205,which will be described later.

The post-processor 205 performs the back mixing processing (mixingrelease processing) on the moving image data Q at the high frame rate,which has been received by the HDMI reception unit 208, and obtains themixing-released moving image data R at the high frame rate under thecontrol by the control unit 201-2A.

The MCFI unit 206 performs frame interpolation processing of motioncompensation on the moving image data R at the high frame rate, whichhas been received by the HDMI reception unit 209 or obtained by thepost-processor 205, and obtains the moving image data at a furtherincreased frame rate. In addition, there is also a case in which thisMCFI unit 206 is not provided. The panel display unit 207 performs imagedisplay based on the moving image data R at the high frame rate that hasbeen obtained by the post-processor 205 or the moving image data at aframe rate increased by the MCFI unit 206.

FIG. 30 illustrates a configuration example of the display 200-2B thatdeals with the moving image data at the normal frame rate. In FIG. 30,the same reference numerals are given to the parts corresponding tothose in FIG. 19. This display 200-2B has a control unit 201-2B, an HDMIreception unit 209B, the MCFI unit 206B, and the panel display unit207B.

The control unit 201-2B controls operations of the respective parts inthe display 200-2B. The HDMI reception unit 209B receives non-compressedmoving image data at the normal frame rate from the set top box 200-1via the HDMI transfer path through the communication in accordance withthe HDMI.

The MICFI unit 206B performs the frame interpolation processing ofmotion compensation on the moving image data at the normal frame rate,which has been received by the HDMI reception unit 209B, and obtains themoving image data at the increased frame rate. In addition, there isalso a case in which the MCFI unit 206B is not provided. The paneldisplay unit 207B performs image display based on the moving image dataat the normal frame rate, which has been received by the HDMI receptionunit 209B, or the moving image data at eh frame rate increased by theMCFI unit 206B.

As described above, in a case in which the moving image data Q at thehigh frame rate, on which the mixing processing has been performed, istransmitted to the display 200-2, the HFR blending infoframe includingthe information about the mixing rate in each frame is simultaneouslytransmitted in the transmission and reception system 10A illustrated inFIG. 21. Therefore, the display 200-2 can easily obtain themixing-released moving image data obtained by performing the back mixingprocessing on the moving image data Q at the high frame rate on thebasis of the information about the mixing rate in each frame and cansatisfactorily displays a moving image.

In addition, although the example in which the high frame rate is 120 Hzor 240 Hz and the normal frame rate is 60 Hz has been illustrated in theaforementioned embodiment, the combination of the frame rates is notlimited thereto. For example, the same is true to the combination of 100Hz or 200 Hz and 50 fps.

In addition, although the transmission and reception system 100 thatincludes the transmission apparatus 100 and the television receiver 200and further the transmission and reception system 10A that includes thetransmission apparatus 100, the set top box 200-1, and the display 200-2have been illustrated in the aforementioned embodiment, theconfiguration of the transmission and reception system to which thepresent technology can be applied is not limited thereto.

In addition, the example in which the container is the transport stream(MPEG-2 TS) has been illustrated in the aforementioned embodiment.However, the present technology can also similarly be applied to asystem with a configuration in which distribution to a receptionterminal is performed by using a network such as the Internet. In theinternet distribution, distribution is performed by using a container ofMP4 or another format in many cases. That is, containers in variousformats such as a transport stream (MPEG-2 TS), or mpeg media transport(MMT) that have been employed by digital broadcasting standards andISOBMFF (MP4) that is used in the Internet distribution can be appliedas the container.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

Additionally, the present technology may also be configured as below.

(1)

A transmission apparatus comprising:

-   -   circuitry configured to    -   perform processing of mixing, at a mixing rate for each frame, a        frame of first video data with one or more peripheral frames of        the first video data and obtain second video data at a first        frame rate,    -   the second video data including frames corresponding to a second        frame rate that is lower than the first frame rate, the frames        corresponding to the second frame rate being mixed with the        peripheral frames, and    -   the circuitry is further configured to    -   encode the frames corresponding to the second frame rate to        obtain a basic stream and encode remaining frames of the second        video data to obtain an extended stream,    -   insert information about the mixing rate of corresponding frames        into the basic stream and the extended stream in association        with the respective frames, and    -   transmit the basic stream and the extended stream into which the        information about the mixing rate has been inserted.        (2)

The transmission apparatus according to (1),

-   -   wherein the basic stream and the extended stream have a Network        Abstraction Layer (NAL) unit structure, and    -   the circuitry is configured to insert a Supplemental Enhancement        Information (SEI) NAL unit with the information about the mixing        rate into the basic stream and the extended stream.        (3)

The transmission apparatus according to (1),

-   -   wherein the first frame rate is 120 Hz or 240 Hz, and the second        frame rate is 60 Hz.        (4)

The transmission apparatus according to (1),

-   -   wherein configuration information of a filter used to perform        the mixing processing is included in the information about the        mixing rate.        (5)

The transmission apparatus according to (1),

-   -   wherein mixing refresh information indicating a number of frames        until mixing refresh that does not use a temporally previous        frame is performed is included in the information about the        mixing rate.        (6)

The transmission apparatus according to (5),

-   -   wherein refresh flag information indicating whether or not a        respective frame is a target of the mixing refresh is included        in the information indicating the mixing rate.        (7)

The transmission apparatus according to (1),

-   -   wherein head flag information indicating whether or not a        respective frame corresponds to the second frame rate is        included in the information about the mixing rate.        (8)

A transmission method comprising:

-   -   performing, by circuitry, processing of mixing, at a mixing rate        for each frame, a frame of first video data with one or more        peripheral frames of the first video data and obtain second        video data at the first frame rate,    -   the second video data including frames corresponding to a second        frame rate that is lower than the first frame rate, the frames        corresponding to the second frame rate being mixed with the        peripheral frames, and    -   the transmission method further includes    -   encoding, by the circuitry, the frames corresponding to the        second frame rate to obtain a basic stream and encoding        remaining frames of the second video data to obtain an extended        stream,    -   inserting, by the circuitry, information about the mixing rate        of corresponding frames into the basic stream and the extended        stream in association with the respective frames, and    -   transmitting, by the circuitry, the basic stream and the        extended stream into which the information about the mixing rate        has been inserted.        (9)

A reception apparatus comprising:

-   -   circuitry configured to receive a basic stream and an extended        stream, which are obtained by    -   performing processing of mixing, at a mixing rate for each        frame, a frame of first video data with one or more peripheral        frames of the first video data and obtaining second video data        at a first frame rate, the second video data including frames        corresponding to a second frame rate that is lower than the        first frame rate, the frames corresponding to the second frame        rate are mixed with the peripheral frames,    -   encoding the frames corresponding to the second frame rate to        obtain the basic stream, and    -   encoding remaining frames of the second video data to obtain the        extended stream, information about the mixing rate of        corresponding frames is included in the basic stream and the        extended stream in association with the respective frames, and    -   the reception apparatus further includes circuitry configured        to, based on a frame rate capability of a display connected to        the reception apparatus,    -   decode the basic stream to obtain frames at the second frame        rate or    -   decode the basic stream and the extended stream to obtain the        second video data, and obtain mixing-released video data at the        first frame rate by performing back mixing processing on the        second video data on a basis of the information about the mixing        rate.        (10)

A reception method comprising:

-   -   receiving, by circuitry, a basic stream and an extended stream,    -   which are obtained by    -   performing processing of mixing, at a mixing rate for each        frame, a frame of first video data with one or more peripheral        frames of the first video data and obtaining second video data        at a first frame rate, the second video data including frames        corresponding to a second frame rate that is lower than the        first frame rate, the frames corresponding to the second frame        rate are mixed with the peripheral frames,    -   encoding the frames corresponding to the second frame rate to        obtain the basic stream, and    -   encoding remaining frames of the second video data to obtain the        extended stream, information about the mixing rate of        corresponding frames is included in the basic stream and the        extended stream in association with the respective frames, and        the reception method further includes, based on a frame rate        capability of a display connected to the reception apparatus,    -   decoding, by the circuitry, the basic stream to obtain frames at        the second frame rate, or    -   decoding the basic stream and the extended stream to obtain the        second video data, and obtaining mixing-released video data at        the first frame rate by performing back mixing processing on the        second video data on a basis of the information about the mixing        rate.        (11)

A reception apparatus comprising:

-   -   circuitry configured to    -   acquire second video data obtained by performing processing of        mixing, at a mixing rate for each frame, a frame of first video        data with one or more peripheral frames of the first video data;        and    -   transmit the second video data and information about the mixing        rate in each frame to an external device via a transfer path.        (12)

The reception apparatus according to (11),

-   -   wherein synchronization frame information indicating whether or        not it is necessary to synchronize with a next video frame is        included in the information about the mixing rate.        (13)

The reception apparatus according to (11),

-   -   wherein the circuitry is configured to respectively insert the        information about the mixing rate in each frame into a blanking        period of each frame of the second video data and transmit the        second video data.        (14)

The reception apparatus according to (11), wherein the circuitry isfurther configured to perform back mixing processing on each frame ofthe second video data on a basis of the information about the mixingrate to obtain third video data,

-   -   wherein the circuitry is configured to transmit the third video        data instead of the second video data when the external device        does not have a function of the back mixing processing.        (15)

The reception apparatus according to (11),

-   -   wherein the second video data has a first frame rate,    -   the second video data including frames corresponding to a second        frame rate that is lower than the first frame rate, the frames        corresponding to the second frame rate are mixed with the        peripheral frames, and    -   the circuitry is further configured to transmit fourth video        data that includes the frames corresponding to the second frame        rate instead of the second video data when a frame rate at which        display is able to be performed by the external device is the        second frame rate.        (16)

A reception method comprising:

-   -   acquiring, by circuitry, second video data obtained by        performing processing of mixing, at a mixing rate for each        frame, a frame of first video data with one or more peripheral        frames of the first video data; and    -   transmitting, by the circuitry, the second video data and        information about the mixing rate in each frame to an external        device via a transfer path.        (17)

A reception apparatus comprising:

-   -   circuitry configured to    -   receive second video data obtained by performing processing of        mixing, at a mixing rate for each frame, a frame of first video        data with one or more peripheral frames of the first video data,        and information about a mixing rate in each frame from an        external device via a transfer path; and    -   obtain mixing-released video data by performing back mixing        processing on each frame of the second video data on a basis of        the information about the mixing rate.        (18)

A reception method comprising:

-   -   receiving, by circuitry, second video data obtained by        performing processing of mixing, at a mixing rate for each        frame, a frame of first video data with one or more peripheral        frames of the first video data, and information about a mixing        rate in each frame from an external device via a transfer path;        and    -   obtaining, by the circuitry, mixing-released video data by        performing back mixing processing on each frame of the second        video data on a basis of the information about the mixing rate.

The main features of the present technology is that it is possible toeasily realize smooth image display by mixing at least the image data inthe frame corresponding to the normal frame rate in the image data ineach frame that forms the moving image data at the high frame rate withthe image data in the peripheral frames to obtain the state in which theaperture ratio is increased, transmitting the basic stream STb obtainedby encoding the image data in the frame corresponding to the normalframe rate, and on the reception side, processing the basic stream andobtaining the moving image data at the normal frame rate (see FIG. 4).In addition, it is also possible to easily obtain the mixing-releasedmoving image data at the high frame rate and to satisfactorily displaythe moving image at the high frame rate on the reception side byobtaining the extended stream STe obtained by encoding the image data inthe residual frames along with the basic stream STb, inserting theinformation (coefficient set) about the mixing rate in the frames intothe basic stream STb and the extended stream STe, and transmitting theinformation therewith (see FIG. 4).

REFERENCE SIGNS LIST

10, 10A transmission and reception system

81 camera

82 HFR processor

100 transmission apparatus

101 control unit

102 pre-processor

102 a, 102 b delay element

102 c, 102 c, 102 d coefficient device

102 f adder

102 g frame output switching device

103 encoder

104 multiplexer

105 transmission unit

200, 200A, 200B television receiver

200-1 set top box

200-2, 200-2A, 200-2B display

201, 201-1, 201-2A, 201-2B control unit

202, 202B reception unit

203, 203B demultiplexer

204, 204B decoder

205 post-processor

205 a, 205 b delay element

205 c, 205 c, 205 d coefficient device

205 f adder

205 g frame output switching device

206, 206B MCFI unit

207, 207B panel display unit

208, 208B HDMI transmission unit

209 HDMI reception unit

1. A transmission apparatus comprising: a processing unit that performsprocessing of mixing, at a mixing rate independent for each frame, imagedata in peripheral frames with image data in each frame of first movingimage data at a first frame rate and obtains second moving image data atthe first frame rate, wherein at least image data in a framecorresponding to a second frame rate that is lower than the first framerate in the image data in each frame that forms the second moving imagedata is brought into a state in which the image data is mixed with theimage data in the peripheral frames, and the transmission apparatusfurther includes an encoding unit that encodes the image data in theframe corresponding to the second frame rate to obtain a basic streamand encodes image data in residual frames to obtain an extended stream,an insertion unit that inserts information about the mixing rate ofcorresponding frames into the basic stream and the extended stream inassociation with the image data in the respective frames, and atransmission unit that transmits a container including the basic streamand the extended stream into which the information about the mixing ratehas been inserted.
 2. The transmission apparatus according to claim 1,wherein the basic stream and the extended stream have a NAL unitstructure, and the insertion unit inserts a SEI NAL unit with theinformation about the mixing rate into the basic stream and the extendedstream.
 3. The transmission apparatus according to claim 1, wherein thefirst frame rate is 120 Hz or 240 Hz, and the second frame rate is 60Hz.
 4. The transmission apparatus according to claim 1, whereinconfiguration information of a filter used to perform the mixingprocessing has been added to the information about the mixing rate. 5.The transmission apparatus according to claim 1, wherein mixing refreshinformation indicating a number of frames until mixing refresh that doesnot use image data in a temporally previous frame is performed has beenadded to the information about the mixing rate.
 6. The transmissionapparatus according to claim 5, wherein refresh flag informationindicating whether or not the frame is a target of the mixing refreshhas been added to the information indicating the mixing rate.
 7. Thetransmission apparatus according to claim 1, wherein head flaginformation indicating whether or not the frame corresponds to thesecond frame rate has been added to the information about the mixingrate.
 8. (canceled)
 9. A reception apparatus comprising: a receptionunit that receives a container including a basic stream and an extendedstream, wherein the basic stream is obtained by encoding image data in aframe corresponding to a second frame rate that is lower than a firstframe rate, which is image data in a state in which the image data ismixed with at least image data in peripheral frames, in image data thatforms second moving image data at the first frame rate, which isobtained by performing processing of mixing, at a mixing rateindependent for each frame, image data in the peripheral frames withimage data in each frame of first moving image data at the first framerate, and the extended stream is obtained by encoding image data inresidual frames, information about the mixing rate of correspondingframes is inserted to the basic stream and the extended stream inassociation with image data in the respective frames, and the receptionapparatus further includes a processing unit that decodes the basicstream to obtain moving image data at the second frame rate or decodesthe basic stream and the extended stream to obtain the second movingimage data in accordance with display ability, and obtainsmixing-released moving image data at the first frame rate by performingback mixing processing on the second moving image data on a basis of theinformation about the mixing rate.
 10. A reception method comprising:receiving, by a reception unit, a container including a basic stream andan extended stream, wherein the basic stream is obtained by encoding atleast image data in a frame corresponding to a second frame rate that islower than a first frame rate, which is image data that has been broughtinto a state in which the image data is mixed with image data inperipheral frames, in image data that forms second moving image data atthe first frame rate, which is obtained by performing processing ofmixing, at a mixing rate independent for each frame, image data in theperipheral frames with image data in each frame of first moving imagedata at the first frame rate, and the extended stream is obtained byencoding image data in residual frames, information about the mixingrate of corresponding frames is inserted to the basic stream and theextended stream in association with image data in the respective frames,and the reception method further includes decoding, by a processingunit, the basic stream to obtain moving image data at the second framerate or decoding the basic stream and the extended stream to obtain thesecond moving image data in accordance with display ability, andobtaining mixing-released moving image data at the first frame rate byperforming back mixing processing on the second moving image data on abasis of the information about the mixing rate. 11-18. (canceled) 19.The reception apparatus according to claim 9, wherein the basic streamand the extended stream have a NAL unit structure, and a SEI NAL unitwith the information about the mixing rate is inserted into the basicstream and the extended stream.
 20. The reception apparatus according toclaim 9, wherein the first frame rate is 120 Hz or 240 Hz, and thesecond frame rate is 60 Hz.
 21. The reception apparatus according toclaim 9, wherein configuration information of a filter used to performthe mixing processing has been added to the information about the mixingrate.
 22. The reception apparatus according to claim 9, wherein mixingrefresh information indicating a number of frames until mixing refreshthat does not use image data in a temporally previous frame is performedhas been added to the information about the mixing rate.
 23. Thereception apparatus according to claim 9, wherein refresh flaginformation indicating whether or not the frame is a target of themixing refresh has been added to the information indicating the mixingrate.
 24. The reception apparatus according to claim 9, wherein headflag information indicating whether or not the frame corresponds to thesecond frame rate has been added to the information about the mixingrate.